Skip to content

Cherry-pick #15227 to 7.x: Add AWS CloudTrail Support#15519

Merged
leehinman merged 1 commit intoelastic:7.xfrom
leehinman:backport_15227_7.x
Jan 14, 2020
Merged

Cherry-pick #15227 to 7.x: Add AWS CloudTrail Support#15519
leehinman merged 1 commit intoelastic:7.xfrom
leehinman:backport_15227_7.x

Conversation

@leehinman
Copy link
Copy Markdown
Contributor

@leehinman leehinman commented Jan 13, 2020

Cherry-pick of PR #15227 to 7.x branch. Original message:

Direct Field Mappings

  • eventTime :: @timestamp
  • eventVersion :: aws.cloudtrail.event_version
  • userIdentity.type :: aws.cloudtrail.user_identity.type
  • userIdentity.userName :: user.name
  • userIdentity.principalId :: user.id
  • userIdentity.arn :: aws.cloudtrail.user_identity.arn
  • userIdentity.accountId :: cloud.account.id
  • userIdentity.accessKeyId :: aws.cloudtrail.user_idenity.access_key_id
  • userIdentity.sessionContext.creation_date :: aws.cloudtrail.user_identity.session_context.creation_date
  • userIdentity.sessionContext.mfa_authenticated :: aws.cloudtrail.user_identity.session_context.mfa_authenticated
  • userIdentity.invocedBy :: aws.cloudtrail.user_identity.invoked_by
  • eventSource :: event.provider
  • eventName :: event.action
  • awsRegion :: cloud.region
  • sourceIPAddress :: source.address
  • userAgent :: user_agent (via user_agent processor)
  • errorCode :: error.code
  • errorMessage :: error.message
  • requestParameters :: aws.cloudtrail.request_parameters (string representation)
  • responseElements :: aws.cloudtrail.response_elements (string representation)
  • requestId :: aws.cloudtrail.request_id
  • eventID :: event.id
  • eventType :: aws.cloudtrail.event_type
  • apiVerison :: aws.cloudtrail.api_version
  • managementEvent :: aws.cloudtrail.management_event
  • readOnly :: aws.cloudtrail.read_only
  • resources.ARN :: aws.cloudtrail.resources.arn
  • resources.accountId :: aws.cloudtrail.resources.account_id
  • resources.type :: aws.cloudtrail.resources.type
  • recipientAccountId :: aws.cloudtrail.recipient_account_id
  • serviceEventDetails :: aws.cloudtrail.service_event_details (string representation)
  • sharedEventId :: aws.cloudtrail.shared_event_id
  • vpcEndpointId :: aws.cloudtrail.vpc_endpoint_id

Other mappings

  • set event.original
  • set event.type
  • set event.kind
  • set event.outcome
  • populate related.users array

To Do

  • move aws.cloudtrail.related.users to related.users when ECS is upgraded to 1.4

@leehinman
[Filebeat] Add AWS CloudTrail Support (elastic#15227)
600d622 
- maps all fields in CloudTrail events
- requestParameters, responseElements, additionalEventData
  & serviceEventDetails are string representations
- add event.original
- add event.type
- add event.kind
- add event.outcome
- run geoip processor
- run agent processor
- populated related.user array when possible
- uses s3input
- CloudTrail must write to S3 bucket, and send all Create Events
  to an SQS queue we listen to

Fixes elastic#14657

(cherry picked from commit da7a697)
@leehinman leehinman requested a review from a team as a code owner January 13, 2020 22:04
@leehinman
Copy link
Copy Markdown
Contributor Author

leehinman commented Jan 14, 2020

The beats-ci failures in filebeat are due to #15486 not being merged yet. The related.user field is in ECS 1.4

@leehinman leehinman merged commit ee82c1f into elastic:7.x Jan 14, 2020
@leehinman leehinman deleted the backport_15227_7.x branch March 27, 2020 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@kaiyan-sheng kaiyan-sheng kaiyan-sheng approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

backport review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@leehinman @kaiyan-sheng